{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 读取文档"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载 PDF\n",
    "loaders_chinese = [\n",
    "    PyPDFLoader(\"../data/第一回：Matplotlib初相识.pdf\"),\n",
    "    PyPDFLoader(\"../data/第一回：Matplotlib初相识.pdf\"),\n",
    "    PyPDFLoader(\"../data/第二回：艺术画笔见乾坤.pdf\"),\n",
    "    PyPDFLoader(\"../data/第三回：布局格式定方圆.pdf\")\n",
    "]\n",
    "docs = []\n",
    "for loader in loaders_chinese:\n",
    "    docs.extend(loader.load())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "29\n"
     ]
    }
   ],
   "source": [
    "# 分割文本\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=1500,\n",
    "    chunk_overlap=150\n",
    ")\n",
    "splits = text_splitter.split_documents(docs)\n",
    "print(len(splits))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('第⼀回： Matplotlib 初相识\\n'\n",
      " '⼀、认识 matplotlib \\n'\n",
      " 'Matplotlib 是⼀个 Python 2D 绘图库，能够以多种硬拷⻉格式和跨平台的交互式环境⽣成出版物质量的图形，⽤来绘制各种\\n'\n",
      " '静态，动态，交互式的图表。\\n'\n",
      " 'Matplotlib 可⽤于 Python 脚本， Python 和 IPython Shell 、 Jupyter notebook ， Web '\n",
      " '应⽤程序服务器和各种图形⽤⼾界⾯⼯具\\n'\n",
      " '包等。\\n'\n",
      " 'Matplotlib 是 Python 数据可视化库中的泰⽃，它已经成为 python 中公认的数据可视化⼯具，我们所熟知的 pandas 和 '\n",
      " 'seaborn\\n'\n",
      " '的绘图接⼝其实也是基于 matplotlib 所作的⾼级封装。\\n'\n",
      " '为了对 matplotlib 有更好的理解，让我们从⼀些最基本的概念开始认识它，再逐渐过渡到⼀些⾼级技巧中。\\n'\n",
      " '⼆、⼀个最简单的绘图例⼦\\n'\n",
      " 'Matplotlib 的图像是画在 figure （如 windows ， jupyter 窗体）上的，每⼀个 figure ⼜包含了⼀个或多个 '\n",
      " 'axes （⼀个可以指定坐\\n'\n",
      " '标系的⼦区域）。最简单的创建 figure 以及 axes 的⽅式是通过pyplot.subplots命令，创建 axes '\n",
      " '以后，可以使⽤Axes.plot绘制\\n'\n",
      " '最简易的折线图。\\n'\n",
      " 'import matplotlib.pyplot as plt\\n'\n",
      " 'import matplotlib as mpl\\n'\n",
      " 'import numpy as np\\n'\n",
      " 'fig, ax = plt.subplots()  # 创建一个包含一个 axes 的 figure\\n'\n",
      " 'ax.plot([1, 2, 3, 4], [1, 4, 2, 3]);  # 绘制图像\\n'\n",
      " 'T r ick ：  在 jupyter notebook 中使⽤ matplotlib '\n",
      " '时会发现，代码运⾏后⾃动打印出类似<matplotlib.lines.Line2D at\\n'\n",
      " '0x23155916dc0>这样⼀段话，这是因为 matplotlib 的绘图代码默认打印出最后⼀个对象。如果不想显⽰这句话，有以下三种\\n'\n",
      " '⽅法，在本章节的代码⽰例中你能找到这三种⽅法的使⽤。\\n'\n",
      " '1. 在代码块最后加⼀个分号;\\n'\n",
      " '2. 在代码块最后加⼀句 plt.show()\\n'\n",
      " '3. 在绘图时将绘图对象显式赋值给⼀个变量，如将 plt.plot([1, 2, 3, 4]) 改成 line =plt.plot([1, 2, 3, '\n",
      " '4])\\n'\n",
      " '和 MATLAB 命令类似，你还可以通过⼀种更简单的⽅式绘制图像，matplotlib.pyplot⽅法能够直接在当前 axes 上绘制图像，\\n'\n",
      " '如果⽤⼾未指定 axes ， matplotlib 会帮你⾃动创建⼀个。所以上⾯的例⼦也可以简化为以下这⼀⾏代码。\\n'\n",
      " 'line =plt.plot([1, 2, 3, 4], [1, 4, 2, 3]) \\n'\n",
      " '\\uf03a Contents \\n'\n",
      " '⼀、认识 matplotlib\\n'\n",
      " '⼆、⼀个最简单的绘图例⼦\\n'\n",
      " '三、 Figure 的组成\\n'\n",
      " '四、两种绘图接⼝\\n'\n",
      " '五、通⽤绘图模板\\n'\n",
      " '思考题\\n'\n",
      " 'P r i n t  t o P D F')\n"
     ]
    }
   ],
   "source": [
    "pprint(splits[0].page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.embeddings.dashscope import DashScopeEmbeddings\n",
    "from dotenv import load_dotenv\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv()\n",
    "api_key = os.environ.get(\"DASHSCOPE_API_KEY\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = DashScopeEmbeddings(dashscope_api_key=api_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "s1 = \"我喜欢狗\"\n",
    "s2 = \"我喜欢犬科动物\"\n",
    "s3 = \"外面的天气很糟糕\"\n",
    "\n",
    "embedding1 = embeddings.embed_query(s1)\n",
    "embedding2 = embeddings.embed_query(s2)\n",
    "embedding3 = embeddings.embed_query(s3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(7171.816694494791)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(embedding1, embedding2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(2307.882459829324)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(embedding1, embedding3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(1321.5981803991517)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(embedding2, embedding3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vectorstroes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.vectorstores import Chroma\n",
    "persist_directory_chinese = './docs/chroma/matplotlib'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "vectordb_chinese = Chroma.from_documents(\n",
    "    documents=splits,\n",
    "    embedding=embeddings,\n",
    "    persist_directory=persist_directory_chinese\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectordb_chinese._collection.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 相似性搜索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "question_chinese = \"Matplotlib是什么？\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "docs_chinese = vectordb_chinese.similarity_search(question_chinese,k=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(docs_chinese)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('第⼀回： Matplotlib 初相识\\n'\n",
      " '⼀、认识 matplotlib \\n'\n",
      " 'Matplotlib 是⼀个 Python 2D 绘图库，能够以多种硬拷⻉格式和跨平台的交互式环境⽣成出版物质量的图形，⽤来绘制各种\\n'\n",
      " '静态，动态，交互式的图表。\\n'\n",
      " 'Matplotlib 可⽤于 Python 脚本， Python 和 IPython Shell 、 Jupyter notebook ， Web '\n",
      " '应⽤程序服务器和各种图形⽤⼾界⾯⼯具\\n'\n",
      " '包等。\\n'\n",
      " 'Matplotlib 是 Python 数据可视化库中的泰⽃，它已经成为 python 中公认的数据可视化⼯具，我们所熟知的 pandas 和 '\n",
      " 'seaborn\\n'\n",
      " '的绘图接⼝其实也是基于 matplotlib 所作的⾼级封装。\\n'\n",
      " '为了对 matplotlib 有更好的理解，让我们从⼀些最基本的概念开始认识它，再逐渐过渡到⼀些⾼级技巧中。\\n'\n",
      " '⼆、⼀个最简单的绘图例⼦\\n'\n",
      " 'Matplotlib 的图像是画在 figure （如 windows ， jupyter 窗体）上的，每⼀个 figure ⼜包含了⼀个或多个 '\n",
      " 'axes （⼀个可以指定坐\\n'\n",
      " '标系的⼦区域）。最简单的创建 figure 以及 axes 的⽅式是通过pyplot.subplots命令，创建 axes '\n",
      " '以后，可以使⽤Axes.plot绘制\\n'\n",
      " '最简易的折线图。\\n'\n",
      " 'import matplotlib.pyplot as plt\\n'\n",
      " 'import matplotlib as mpl\\n'\n",
      " 'import numpy as np\\n'\n",
      " 'fig, ax = plt.subplots()  # 创建一个包含一个 axes 的 figure\\n'\n",
      " 'ax.plot([1, 2, 3, 4], [1, 4, 2, 3]);  # 绘制图像\\n'\n",
      " 'T r ick ：  在 jupyter notebook 中使⽤ matplotlib '\n",
      " '时会发现，代码运⾏后⾃动打印出类似<matplotlib.lines.Line2D at\\n'\n",
      " '0x23155916dc0>这样⼀段话，这是因为 matplotlib 的绘图代码默认打印出最后⼀个对象。如果不想显⽰这句话，有以下三种\\n'\n",
      " '⽅法，在本章节的代码⽰例中你能找到这三种⽅法的使⽤。\\n'\n",
      " '1. 在代码块最后加⼀个分号;\\n'\n",
      " '2. 在代码块最后加⼀句 plt.show()\\n'\n",
      " '3. 在绘图时将绘图对象显式赋值给⼀个变量，如将 plt.plot([1, 2, 3, 4]) 改成 line =plt.plot([1, 2, 3, '\n",
      " '4])\\n'\n",
      " '和 MATLAB 命令类似，你还可以通过⼀种更简单的⽅式绘制图像，matplotlib.pyplot⽅法能够直接在当前 axes 上绘制图像，\\n'\n",
      " '如果⽤⼾未指定 axes ， matplotlib 会帮你⾃动创建⼀个。所以上⾯的例⼦也可以简化为以下这⼀⾏代码。\\n'\n",
      " 'line =plt.plot([1, 2, 3, 4], [1, 4, 2, 3]) \\n'\n",
      " '\\uf03a Contents \\n'\n",
      " '⼀、认识 matplotlib\\n'\n",
      " '⼆、⼀个最简单的绘图例⼦\\n'\n",
      " '三、 Figure 的组成\\n'\n",
      " '四、两种绘图接⼝\\n'\n",
      " '五、通⽤绘图模板\\n'\n",
      " '思考题\\n'\n",
      " 'P r i n t  t o P D F')\n"
     ]
    }
   ],
   "source": [
    "pprint(docs_chinese[0].page_content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('第⼀回： Matplotlib 初相识\\n'\n",
      " '⼀、认识 matplotlib \\n'\n",
      " 'Matplotlib 是⼀个 Python 2D 绘图库，能够以多种硬拷⻉格式和跨平台的交互式环境⽣成出版物质量的图形，⽤来绘制各种\\n'\n",
      " '静态，动态，交互式的图表。\\n'\n",
      " 'Matplotlib 可⽤于 Python 脚本， Python 和 IPython Shell 、 Jupyter notebook ， Web '\n",
      " '应⽤程序服务器和各种图形⽤⼾界⾯⼯具\\n'\n",
      " '包等。\\n'\n",
      " 'Matplotlib 是 Python 数据可视化库中的泰⽃，它已经成为 python 中公认的数据可视化⼯具，我们所熟知的 pandas 和 '\n",
      " 'seaborn\\n'\n",
      " '的绘图接⼝其实也是基于 matplotlib 所作的⾼级封装。\\n'\n",
      " '为了对 matplotlib 有更好的理解，让我们从⼀些最基本的概念开始认识它，再逐渐过渡到⼀些⾼级技巧中。\\n'\n",
      " '⼆、⼀个最简单的绘图例⼦\\n'\n",
      " 'Matplotlib 的图像是画在 figure （如 windows ， jupyter 窗体）上的，每⼀个 figure ⼜包含了⼀个或多个 '\n",
      " 'axes （⼀个可以指定坐\\n'\n",
      " '标系的⼦区域）。最简单的创建 figure 以及 axes 的⽅式是通过pyplot.subplots命令，创建 axes '\n",
      " '以后，可以使⽤Axes.plot绘制\\n'\n",
      " '最简易的折线图。\\n'\n",
      " 'import matplotlib.pyplot as plt\\n'\n",
      " 'import matplotlib as mpl\\n'\n",
      " 'import numpy as np\\n'\n",
      " 'fig, ax = plt.subplots()  # 创建一个包含一个 axes 的 figure\\n'\n",
      " 'ax.plot([1, 2, 3, 4], [1, 4, 2, 3]);  # 绘制图像\\n'\n",
      " 'T r ick ：  在 jupyter notebook 中使⽤ matplotlib '\n",
      " '时会发现，代码运⾏后⾃动打印出类似<matplotlib.lines.Line2D at\\n'\n",
      " '0x23155916dc0>这样⼀段话，这是因为 matplotlib 的绘图代码默认打印出最后⼀个对象。如果不想显⽰这句话，有以下三种\\n'\n",
      " '⽅法，在本章节的代码⽰例中你能找到这三种⽅法的使⽤。\\n'\n",
      " '1. 在代码块最后加⼀个分号;\\n'\n",
      " '2. 在代码块最后加⼀句 plt.show()\\n'\n",
      " '3. 在绘图时将绘图对象显式赋值给⼀个变量，如将 plt.plot([1, 2, 3, 4]) 改成 line =plt.plot([1, 2, 3, '\n",
      " '4])\\n'\n",
      " '和 MATLAB 命令类似，你还可以通过⼀种更简单的⽅式绘制图像，matplotlib.pyplot⽅法能够直接在当前 axes 上绘制图像，\\n'\n",
      " '如果⽤⼾未指定 axes ， matplotlib 会帮你⾃动创建⼀个。所以上⾯的例⼦也可以简化为以下这⼀⾏代码。\\n'\n",
      " 'line =plt.plot([1, 2, 3, 4], [1, 4, 2, 3]) \\n'\n",
      " '\\uf03a Contents \\n'\n",
      " '⼀、认识 matplotlib\\n'\n",
      " '⼆、⼀个最简单的绘图例⼦\\n'\n",
      " '三、 Figure 的组成\\n'\n",
      " '四、两种绘图接⼝\\n'\n",
      " '五、通⽤绘图模板\\n'\n",
      " '思考题\\n'\n",
      " 'P r i n t  t o P D F')\n"
     ]
    }
   ],
   "source": [
    "pprint(docs_chinese[1].page_content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('第⼆回：艺术画笔⻅乾坤\\n'\n",
      " 'import numpy as np\\n'\n",
      " 'import pandas as pd\\n'\n",
      " 'import re\\n'\n",
      " 'import matplotlib\\n'\n",
      " 'import matplotlib.pyplot as plt\\n'\n",
      " 'from matplotlib.lines import Line2D   \\n'\n",
      " 'from matplotlib.patches import Circle, Wedge\\n'\n",
      " 'from matplotlib.collections import PatchCollection\\n'\n",
      " '⼀、概述\\n'\n",
      " '1. matplotlib 的三层 api\\n'\n",
      " 'matplotlib 的原理或者说基础逻辑是，⽤ Artist 对象在画布 (canvas) 上绘制 (Render) 图形。\\n'\n",
      " '就和⼈作画的步骤类似：\\n'\n",
      " '1. 准备⼀块画布或画纸\\n'\n",
      " '2. 准备好颜料、画笔等制图⼯具\\n'\n",
      " '3. 作画\\n'\n",
      " '所以 matplotlib 有三个层次的 API ：\\n'\n",
      " 'matplotlib.backend_bases.FigureCanvas 代表了绘图区，所有的图像都是在绘图区完成的\\n'\n",
      " 'matplotlib.backend_bases.Renderer 代表了渲染器，可以近似理解为画笔，控制如何在  FigureCanvas 上画图。\\n'\n",
      " 'matplotlib.artist.Artist 代表了具体的图表组件，即调⽤了 Renderer 的接⼝在 Canvas 上作图。\\n'\n",
      " '前两者处理程序和计算机的底层交互的事项，第三项 Artist 就是具体的调⽤接⼝来做出我们想要的图，⽐如图形、⽂本、线\\n'\n",
      " '条的设定。所以通常来说，我们 95% 的时间，都是⽤来和 matplotlib.artist.Artist 类打交道的。\\n'\n",
      " '2. Artist 的分类\\n'\n",
      " 'Artist 有两种类型：primitives 和containers。\\n'\n",
      " 'primitive是基本要素，它包含⼀些我们要在绘图区作图⽤到的标准图形对象，如曲线 Line2D ，⽂字 text ，矩形\\n'\n",
      " 'Rectangle ，图像 image等。\\n'\n",
      " 'container是容器，即⽤来装基本要素的地⽅，包括图形 figure 、坐标系 Axes 和坐标轴 Axis。他们之间的关系如下图所⽰：\\n'\n",
      " '分类\\n'\n",
      " '可视化中常⻅的 artist 类可以参考下图这张表格，解释下每⼀列的含义。\\n'\n",
      " '第⼀列表⽰ matplotlib 中⼦图上的辅助⽅法，可以理解为可视化中不同种类的图表类型，如柱状图，折线图，直⽅图等，\\n'\n",
      " '这些图表都可以⽤这些辅助⽅法直接画出来，属于更⾼层级的抽象。\\n'\n",
      " '第⼆列表⽰不同图表背后的 artist 类，⽐如折线图⽅法plot在底层⽤到的就是Line2D这⼀ artist 类。\\n'\n",
      " '第三列是第⼆列的列表容器，例如所有在⼦图中创建的Line2D对象都会被⾃动收集到ax.lines返回的列表中。\\n'\n",
      " '下⼀节的具体案例更清楚地阐释了这三者的关系，其实在很多时候，我们只⽤记住第⼀列的辅助⽅法进⾏绘图即可，⽽⽆\\n'\n",
      " '需关注具体底层使⽤了哪些类，但是了解底层类有助于我们绘制⼀些复杂的图表，因此也很有必要了解。\\n'\n",
      " 'P r i n t  t o P D F')\n"
     ]
    }
   ],
   "source": [
    "pprint(docs_chinese[2].page_content)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
